Spiking neural networks (SNNs) are receiving increasing attention due to their low power consumption and strong bio-plausibility. Optimization of SNNs is a challenging task. Two main methods, artificial neural network (ANN)-to-SNN conversion and spike-based backpropagation (BP), both have their advantages and limitations. For ANN-to-SNN conversion, it requires a long inference time to approximate the accuracy of ANN, thus diminishing the benefits of SNN. With spike-based BP, training high-precision SNNs typically consumes dozens of times more computational resources and time than their ANN counterparts. In this paper, we propose a novel SNN training approach that combines the benefits of the two methods. We first train a single-step SNN by approximating the neural potential distribution with random noise, then convert the single-step SNN to a multi-step SNN losslessly. The introduction of Gaussian distributed noise leads to a significant gain in accuracy after conversion. The results show that our method considerably reduces the training and inference times of SNNs while maintaining their high accuracy. Compared to the previous two methods, ours can reduce training time by 65%-75% and achieves more than 100 times faster inference speed. We also argue that the neuron model augmented with noise makes it more bio-plausible.

Sparse representation has attracted great attention because it can greatly save storage re- sources and find representative features of data in a low-dimensional space. As a result, it may be widely applied in engineering domains including feature extraction, compressed sensing, signal denoising, picture clustering, and dictionary learning, just to name a few. In this paper, we propose a spiking sampling network. This network is composed of spiking neurons, and it can dynamically decide which pixel points should be retained and which ones need to be masked according to the input. Our experiments demonstrate that this approach enables better sparse representation of the original image and facilitates image reconstruction compared to random sampling. We thus use this approach for compressing massive data from the dynamic vision sensor, which greatly reduces the storage requirements for event data.

Spiking neural networks (SNNs) attract great attention due to their low power consumption, low latency, and biological plausibility. As they are widely deployed in neuromorphic devices for low-power brain-inspired computing, security issues become increasingly important. However, compared to deep neural networks (DNNs), SNNs currently lack specifically designed defense methods against adversarial attacks. Inspired by neural membrane potential oscillation, we propose a novel neural model that incorporates the bio-inspired oscillation mechanism to enhance the security of SNNs. Our experiments show that SNNs with neural oscillation neurons have better resistance to adversarial attacks than ordinary SNNs with LIF neurons on kinds of architectures and datasets. Furthermore, we propose a defense method that changes model's gradients by replacing the form of oscillation, which hides the original training gradients and confuses the attacker into using gradients of 'fake' neurons to generate invalid adversarial samples. Our experiments suggest that the proposed defense method can effectively resist both single-step and iterative attacks with comparable defense effectiveness and much less computational costs than adversarial training methods on DNNs. To the best of our knowledge, this is the first work that establishes adversarial defense through masking surrogate gradients on SNNs.

Earth observation, aiming at monitoring the state of planet Earth using remote sensing data, is critical for improving our daily lives and living environment. With a growing number of satellites in orbit, an increasing number of datasets with diverse sensors and research domains are being published to facilitate the research of the remote sensing community. In this paper, we present a comprehensive review of more than 400 publicly published datasets, including applications like land use/cover, change/disaster monitoring, scene understanding, agriculture, climate change, and weather forecasting. We systematically analyze these Earth observation datasets with respect to five aspects volume, bibliometric analysis, resolution distributions, research domains, and the correlation between datasets. Based on the dataset attributes, we propose to measure, rank, and select datasets to build a new benchmark for model evaluation. Furthermore, a new platform for Earth observation, termed EarthNets, is released as a means of achieving a fair and consistent evaluation of deep learning methods on remote sensing data. EarthNets supports standard dataset libraries and cutting-edge deep learning models to bridge the gap between the remote sensing and machine learning communities. Based on this platform, extensive deep learning methods are evaluated on the new benchmark. The insightful results are beneficial to future research. The platform and dataset collections are publicly available at https://earthnets.github.io/.

这里介绍了人工智能研究所（IARAI）组织的2022年Landslide4sense（L4S）竞赛的科学结果。竞争的目的是根据全球收集的卫星图像的大规模多个来源自动检测滑坡。 2022 L4S旨在促进有关使用卫星图像的语义分割任务的深度学习模型（DL）模型最新发展的跨学科研究。在过去的几年中，由于卷积神经网络（CNN）的发展，基于DL的模型已经达到了对图像解释的期望。本文的主要目的是介绍本次比赛中介绍的细节和表现最佳的算法。获胜的解决方案详细介绍了Swin Transformer，Segformer和U-NET等最先进的模型。还考虑了先进的机器学习技术和诸如硬采矿，自我培训和混合数据增强之类的策略。此外，我们描述了L4S基准数据集，以促进进一步的比较，并在线报告准确性评估的结果。可以在\ textIt {未来开发排行榜上访问数据，以供将来评估，\ url {https://www.iarai.ac.ac.at/landslide4sense/challenge/}，并邀请研究人员提交更多预测结果，评估准确性在他们的方法中，将它们与其他用户的方法进行比较，理想情况下，改善了本文报告的滑坡检测结果。

通过深度学习技术的开花，完全有监督的基于骨架的动作识别取得了巨大进步。但是，这些方法需要足够的标记数据，这不容易获得。相比之下，基于自我监督的骨骼的动作识别引起了更多的关注。通过利用未标记的数据，可以学会更多可概括的功能来减轻过度拟合的问题并减少大规模标记的培训数据的需求。受到MAE的启发，我们提出了一个空间式蒙面的自动编码器框架，用于基于3D骨架的自我监管的动作识别（Skeletonmae）。在MAE的掩蔽和重建管道之后，我们利用基于骨架的编码器变压器体系结构来重建蒙版的骨架序列。一种新颖的掩蔽策略，称为时空掩蔽，是根据骨架序列的联合级别和框架级别引入的。这种预训练策略使编码器输出可推广的骨骼特征具有空间和时间依赖性。给定未掩盖的骨架序列，编码器用于动作识别任务。广泛的实验表明，我们的骨架达到了出色的性能，并优于NTU RGB+D和NTU RGB+D 120数据集的最新方法。

以人为本的人工智能考虑了人工智能表现的经验。尽管丰富的研究一直在通过全自动或弱监督学习来帮助AI实现超人类的表现，但较少的努力正在尝试AI如何量身定制人类对人类首选技能水平的限制。在这项工作中，我们指导课程加强学习结果朝着首选的绩效水平，通过从人类的决策过程中学习而不是太困难也不容易。为了实现这一目标，我们开发了一个便携式交互式平台，使用户能够通过操纵任务难度，观察性能并提供课程反馈来在线与代理商进行交互。我们的系统高度可行，使人类可以训练大规模的增强学习应用程序，这些学习应用需要数百万没有服务器的样品。结果证明了互动课程对涉及人类在环的增强学习的有效性。它显示强化学习绩效可以成功地与人类所需的难度水平同步调整。我们认为，这项研究将为实现流动和个性化的适应性困难打开新的大门。

移动设备以及它们生成的大量数据是基于机器学习（ML）的应用程序的关键推动者。传统的ML技术已转向新的范式，例如联合（FL）和分裂学习（SL），以改善对用户数据隐私的保护。但是，这些范式通常依靠位于边缘或云中的服务器来训练ML模型的计算重量部分，以避免在客户端设备上排出有限的资源，从而将设备数据暴露于此类第三方。这项工作提出了一种替代方法，可以在用户设备本身中培训计算重量的ML模型，该模型位于相应的设备数据所在的地方。具体来说，我们专注于gan（生成对抗网络），并利用其固有的隐私保护属性。我们在用户的设备上使用原始数据训练GAN的判别部分，而生成模型进行了远程训练（例如服务器），无需访问传感器真实数据。此外，我们的方法可确保训练的计算负载判别模型在用户的设备中共享与SL的计算功能。我们在实际资源约束设备中实施了计算重的GAN模型的建议协作培训计划。结果表明，我们的系统可以保留数据隐私，保持短暂的训练时间，并在不受约束的设备（例如云）中产生相同的模型培训准确性。我们的代码可以在https://github.com/yukarisonz/fsl-gan上找到

可以通过合成孔径雷达（SAR）图像来缓解云去除任务的挑战，这些图像可以穿透云覆盖。但是，光学图像和SAR图像之间的较大域间隙以及SAR图像的严重斑点噪声可能会导致基于SAR的基于SAR的云去除，从而导致性能退化。在本文中，我们提出了一种新型的基于全局融合的云去除（GLF-CR）算法，以利用SAR图像中嵌入的互补信息。利用SAR信息的力量促进云清除需要两个方面。首先是全球融合，指导所有本地光窗口之间的关系，以维持与其余无云区域一致的回收区域的结构。第二个本地融合，传输嵌入在SAR图像中的互补信息，该信息与多云区域相对应，以生成缺失区域的可靠纹理细节，并使用动态过滤来减轻斑点噪声引起的性能退化。广泛的评估表明，所提出的算法可以产生高质量的无云图像，并且在SEN12MS-CR数据集中的PSNR方面，其增益约为1.7 db，超过最先进的云去除算法。

作为计算机视觉的重要领域，对象跟踪形成了两个独立的社区，分别研究单个对象跟踪（SOT）和多个对象跟踪（MOT）。但是，由于两个任务的不同训练数据集和跟踪对象，因此在一个跟踪方案中的当前方法不容易适应另一种方法。尽管unitrack \ cite {wang2021Diverent}表明，具有多个头部的共享外观模型可用于处理单个跟踪任务，但它无法利用大规模跟踪数据集进行训练，并且在单个对象跟踪上执行良好的训练。在这项工作中，我们提出了统一的变压器跟踪器（UTT），以通过一个范式在不同方案中解决跟踪问题。在我们的UTT中开发了轨道变压器，以跟踪SOT和MOT中的目标。利用目标和跟踪框架功能之间的相关性以定位目标。我们证明SOT和MOT任务都可以在此框架内解决。该模型可以同时通过在单个任务数据集中优化SOT和MOT目标，同时端到端训练。广泛的实验是在几个基准测试基准上进行的，该基准具有在SOT和MOT数据集上训练的统一模型。代码将在https://github.com/flowerfan/trackron上找到。